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Abstract 


\ ) That do students expect before starting a Virtual Exchange 

(VE) with peers? Are their initial expectations mirrored in 
the final outcomes they perceive after the VE experience? Or, else, 
do students acknowledge benefits and acquired skills which they did 
not expect at first? This study draws on qualitative and quantitative 
data collected across a variety of VE settings within the Erasmus+ 
EVOLVE project (16 VEs and 248 students in total). In a mixed- 
methods approach, it confronts students’ expectations and perceived 
outcomes in order to outline the learning potential of VE, beyond the 
specific learning objectives set in the different Higher Educational 
(HE) courses in which a VE was implemented. It brings to light that 
the overlap between students’ expectations and the benefits they 
see is only partial. The declared outcomes deviate more from the 
course learning objectives than the initial expectations do, and they 
are, unsurprisingly, more nuanced and manifold. Expectations of 
intercultural and language practice and skills development are more 
often aligned with outcomes voiced by students than is the case for 
digital and disciplinary skills. What stands out is a great occurrence 


of transferable skills in the outcomes, including collaborative, 
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relational, and communicative skills that are not always promoted 


in the course objectives. 


Keywords: virtual exchange, learning outcomes, student expectations, course 


objectives. 


4: Background to the research 


Since telecollaborative projects and partnerships between geographically 
distant groups began to penetrate teaching practices almost three decades ago, 
VE - in its different declinations (O’ Dowd, 2018) —has expanded in HE, being 
implemented in different settings, across a wide range of disciplines and with 
various pedagogical aims (Helm, 2015; O’Dowd & Lewis, 2016). Research 
in this field supports the constantly growing set-up of VE in HE courses 
and contributes to highlighting its impact on the development of students’ 
competences. However, results stem most often from case studies, and if they 
attest to the diversity and richness of practices, they still have a strong context- 
specific scope, and cannot inform about generalizable benefits inherent to the 
VE itself. 


The goal of the Erasmust+t EVOLVE project (EVOLVE website*; Jager et 
al., 2019) is to fill this gap by exploring large-scale data in order to identify 
common trends across a wide range of VEs, and to provide a broader picture 
of the learning potential of VE. Among the different investigations conducted 
within the EVOLVE project, the study presented in this paper focuses on the 
comparison between students’ initial expectations at the outset of the VE, and 
the outcomes and gains stated after completion. The aim is to go beyond the 
measure of students’ satisfaction, and to broaden the results of isolated case 
studies (e.g. Gimeno, 2018; Marczak, 2016). 


4. https://evolve-erasmus.eu 
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2. Methodology for data collection and analysis 
2.1. Data collection 


The data retrieved within EVOLVE stems from 16 VEs with various settings, 
which took place between September 2019 and May 2020, involving 24 HE 
institutions worldwide. The diversity of the VE settings is due to languages used, 
partners or disciplines involved, duration, and targeted outcomes (for a more 
detailed VE description, see EVOLVE Project Team, 2020). 


For this paper, the measure of objective attainment through a five-point Likert 
scale constituted the starting point for our analyzes. In order to gain deeper 
insight into these quantitative results, students’ answers to specific open-ended 
questions in administered pre-and post-surveys (N=248) were analyzed in a 
comprehensive approach; follow-up interviews (N=19) provided additional 
information for the qualitative analysis. Figure | indicates the specific questions 
for the collection of data investigated in this paper: two post-survey questions 
with Likert scales for quantitative data collection, and four open-ended questions 
at pre- and post-stages (post-survey and interview). 


Figure 1. Methods for data collection and specific questions investigated in 
this paper 


® @) @) 


PRE-SURVEY POST-SURVEY INTERVIEW 
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“What do you hope to get out 
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of or learn from this VE?” 


learn from this VE?” you learned from this 
| experience?” 
“The VE tasks met the learning “At the very beginning of your 
(el) ed mem meelierme(miciamm VE, what did you hope to learn 
RYer1(2)) or get out of the VE?" 
“The VE tasks met my personal 


learning objectives” (Likert 


scale) 
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2.2. | Processing of the data: establishing coding 
categories and identifying patterns 


This mixed-methods design for the research (Figure |) allows for elaboration 
on quantitative results (Brown & Coombe, 2015, p. 81). The data generated 
from open-ended questions in the surveys and interview were analyzed using 
the software Nvivo 12 Pro and by adopting an inductive bottom-up approach 
for identifying and defining coding categories for students’ expectations and 
outcomes (Zhang & Wildemuth, 2009). In order to allow for comparison, the 
categories established for coding expectations were — as far as possible — aligned 
with categories for outcomes. 


Across the 16 VEs, 12 categories were established for coding expectations 
as well as outcomes. These include disciplinary, intercultural, language, and 
digital competences, which are frequently examined VE objectives in the field’s 
literature (O’Dowd & Lewis, 2016) and targeted in HE courses. Along with 
these, students report on several transferable skills, on the opportunity to make a 
new experience, or to communicate (in the L2 or more generally), to collaborate 
with their peers, and to build interpersonal relationships. Adaptability occurred 
among the reported outcomes as an additional skill that students had not expected. 


Figure 2. Identified patterns for comparison between declared expectations 
and outcomes 


among their expectations 


1. Initial expectations are Student's indications of VE outcomes match those of 
Y fulfilled their initial expectations 
BD 2. Initial expectations are Several VE outcomes a student indicates match the 
& partially met expectations he/she initially expressed 
3. Initial expectations are None of the VE outcomes a student indicates matches 
@ totally unmet the expectations he/she initially expressed 
A 4. Unforeseen benefits Students indicate outcomes that they did not mention 


Patterns based on 
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Through a preliminary study, four main patterns emerged when confronting 
students’ expressed expectations and outcomes. 


The first three identified patterns (Figure 2) relate to various degrees of 
correspondence between students’ initially declared expectations from the VE 
experience, and what they finally state having learned or got out of it. The fourth 
pattern relates to unexpected outcomes. An additional investigation aimed at 
determining to what extent students’ expectations and outcomes were in line 
with the objectives set for the different courses the VE was integrated in. 


3. Results and discussion 

3.1. Perception of learning 

Students’ level of agreement collected through the Likert scale questions indicate 
an overall positive to very positive assessment regarding the achievement of 
learning objectives through the VE, be it the courses’ learning objectives or their 


own (Figure 3 and Figure 4). 


Figure 3. Perceived learning outcomes regarding the course objectives (five- 
point Likert scale) 
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Figure 4. Perceived learning outcomes regarding the personal objectives 


The VE tasks met my personal learning objectives 
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The coding of open-ended questions enabled us to describe more precisely the 
declared expectations and their alignment with the perceived outcomes — both 
related or not to the courses’ targeted objectives — by assigning categories to 
students’ responses (see 2.2). If the most reported expectation before starting 
the VE concerns cultural or intercultural aspects, a shift can be observed in the 
ranking when students report on outcomes once the VE has ended: cultural 
and intercultural aspects are cited much more frequently, and students value 
collaborative teamwork as an important skill VE helped develop. 


Figure 5. Ranking of mentioned expectations (in percentages) 
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Figure 6. Ranking of declared outcomes (in percentages) 
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After collaborative teamworking skills, it is communication with others — and 
not exclusively the development of L2 competence — which also appears as 
a major outcome. This shows the role the communicative experience plays 
during the exchanges, not only for the achievement of the VE tasks, but also for 
establishing interpersonal relationships. 


Differences between Figure 5 and Figure 6 indicate that the perceived VE 
outcomes exceed the initially stated expectations. The students quote many more 
aspects than they initially expected, and for most of the categories the percentage 
of mentions is higher. 


After comparing expectations mentioned by the students to their declared 
outcomes, three main tendencies stand out, which refer to three of the pre- 
identified patterns (see Figure 2): 


¢ all evoked expectations were fulfilled for 29% of the participants, who 
did not mention having gained other unexpected benefits from their VE 


participation (Pattern 1); 


¢ for31% ofthe students, no initial expectations were fulfilled (Pattern 3); 
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¢ for 39% of the participants, some of the initially reported expectations 
appear in the outcomes (Pattern 2). Such an alignment concerns 
two categories: cultural and intercultural aspects and collaborative 
teamwork. Both categories are, on the one hand, the most frequently met 
expectations and on the other hand, the most quoted unforeseen benefits. 


The proportion of met and unmet categories in each of these three patterns being 
globally the same, it is necessary to identify more generally speaking the most 


frequently met expectations (Figure 7). 


Figure 7. Most frequently met expectations (in percentages) 
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When cultural and intercultural aspects are mentioned in the expectations, 
in 67% of the cases, they appear in the outcomes as well. The expectation of 
collaborative teamwork is also satisfied in 63% of the cases. Other expectations 
are fulfilled in less than 50% of the cases. The results must however be nuanced, 
as some references which have been coded for specific categories (for instance 
L2 competence, opportunity for L2 practice, and communication with others) 
can refer to similar aspects. Slight differences in students’ formulations may 
therefore have led us to attribute their statement of expectation to another 
category than the expressed outcome. 
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Figure 8. Unforeseen benefits (number of cases) 
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Regarding the fourth pattern — unforeseen outcomes (see Figure 8) — 67% of 
the students mention gains they did not voice as an expectation before the 
VE. This highlights the dimension of uncaptured benefits VE might bring to 
the participants. Interestingly, unexpected outcomes refer again to cultural and 
intercultural aspects, but concern also collaborative and communicative skills, 
which are among the most often declared outcomes. 


Engagement in VE provides participants with the opportunity to put their 
knowledge and skills into practice under real conditions of use. Students 
experienced communication with remote distant partners coming from different 
geographical areas, which helped them discover other worldviews and reflect 
on them. They were encouraged to work collaboratively with their peers to 
achieve the assigned tasks and had to cope with possible issues, developing 
adaptability, empathy, and social skills. What numerous students particularly 
valued is the authentic experiential and enjoyable learning provided by the 
VE, which may have led them to expand their initial expectations. This is 
illustrated by the following student statement at post-stage. 
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“T got to improve my level in french and get better with my compositions 
in french. I got the chance to meet new people, to work with people 
at distance. We had to work in groups sometimes which helped us 
improve our teamwork, to learn what other people think and listen to 
their opinions. I really enjoyed this experience”. 


3.2. Perceived outcomes 
that broaden course objectives 


It can be assumed that objectives set for the course and disciplinary contents 
may influence the expectations of the students. Information gathered from the 
different partner teachers has enabled us to identify which were the targeted 
learning objectives for each VE, as well as the learning objectives established 
in each course. For this part of our study, information about the courses run 
by the two (or three) partners involved in a VE has been compiled. It appeared 
that in three VEs, students from one partner institution had not completed the 
pre- and post-survey, so that data for three courses was missing; in five other 
courses only one or two students completed both the pre- and post-survey, 
which was decided not sufficient for considering the course. Consequently, the 
following data was gathered from 26 (out of 34) courses from 15 different VEs 
and for which responses of at least three students were available. 


The learning objectives indicated by teachers referred to language skills, 
intercultural competence, disciplinary knowledge (especially teaching skills and 
critical digital literacy), encompassing either one single type of knowledge or 
skills or a set of competences. As shown in Figure 9, students’ expectations at 
the outset of the VE are not identical to the learning objectives set for the course 
but tend to align in a majority of cases (58.4 %). For 23 out of 26 courses, 
students mention all or more than half of the course objectives as expectations. 
In contrast, the declared outcomes are less in line with the course objectives 
(24.1%) and other benefits gained through the VE experience are stated. These 
results seem to indicate that VE entails a greater potential for developing more or 
other competences and skills than initially expected or planned, be it by students 
or by educators. 
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expectations and outcomes 


Figure 9. Congruence rate between course objectives and students’ declared 


Course objectives vs students' stated expectations and perceived outcomes 
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Figure 10. Proportion of students’ expectations and perceived outcomes 
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Figure 11. Proportion of students’ expectations and perceived outcomes 
congruent with course objectives (disciplinary skills) 


Disciplinary skills (future educators) 


a expectations @ outcomes 


The frequency of coding for the two categories (digital literacy and disciplinary 
skills related to language teaching and learning) indicated in Figure 10 and 
Figure 11 confirm differences of overlap in expectations and outcomes. On the 
one hand, students did not necessarily mention expectations or gains regarding 
digital literacy and valued other aspects, especially relational or collaborative 
ones. But on the other hand, improvement in digital skills was among reported 
outcomes even if this competence was not explicitly targeted as a course 
objective. In the same vein, if disciplinary skills are not always stressed by 
the students as the major outcome of their VE, this is due to the fact that they 
highlighted having gained additional (transversal) skills, such as adaptability, 
collaborative skills, or intercultural communicative competence, as shown in the 
following examples. 


“The most valuable thing of [VEs] is that students would not only get 


profit towards their academic development but mainly towards their 
personal and individual development’. 
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“T learned about the different communication styles of people, how to 
collaborate with others who have different communication styles, and 
how to better respond to others when dealing with conflict”. 


Figure 12.Proportion of students’ expectations and perceived outcomes 
congruent with course objectives (language skills) 


L2 competence and opportunity for L2 practice 
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However, a greater alignment can be observed when language and intercultural 
skills are at play, as VE was in some cases integrated in a language course. 
Regarding language skills — referring to both categories “L2 competence” and 
“opportunity of L2 practice” (Figure 12), the high proficiency level of some 
students may explain why language improvement is not always perceived 
neither as an expectation nor as an outcome. Consequently, those students 
considered other targeted outcomes more important or unforeseen benefits much 
more salient, such as the building of interpersonal relationships, or adaptability. 
The development of intercultural competence was a learning objective in several 
courses (Figure 13); only few students in these courses voiced other expectations 
and if so, they still acknowledged an intercultural benefit among the outcomes 


151 


Chapter 11 


of the VE. The results for these two categories are in line with previous studies, 
which have demonstrated the impact of VE on the development of language 
skills and intercultural competence (e.g. Helm & van der Velden, 2020; The 
EVALUATE group, 2019). Nevertheless, the qualitative analysis of the open- 
ended questions reveals that students’ expectations regarding their learning 
are often very general or narrow, whereas their description of the perceived 
outcomes is often much broader and goes beyond the mere objectives of their 
university course. 


Figure 13. Proportion of students’ expectations and perceived outcomes 
congruent with course objectives (intercultural aspects) 
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VE offers a unique educational context in which participants are likely not 
only to enhance but also to challenge their communicative, collaborative, and 
intercultural skills. In that sense, students may develop transversal skills in 
addition to the disciplinary contents or the learning objectives set in their HE 
course and beyond their own learning expectations as well. Nature and depth of 
personal or academic learning objectives can be modified as a consequence of 
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the VE experience and students’ engagement in VE may therefore contribute to 
the development of more complex and intertwined competences, as illustrated in 
the following student post-survey statement. 


“I got an opportunity to use English in a more authentic way by 
cooperating with people from other countries who also use English 
as a lingua franca. I learned how difficult it is to include seven people 
in a project and make everyones voices heard. I learned more about 
how I function in a group work and how I perceive my country and 
my culture as well as other countries and cultures” (Nissen, Felce, & 
Muller, 2020, p. 46). 


4. Conclusion 


This study brings to light occurring shifts and changes between what students 
expect to get out of or learn from international task-based peer-to-peer 
interaction in a VE and what they finally declare as major VE outcomes. The 
greatest match between expectations and outcomes relates to intercultural or 
cultural aspects. Qualitative insights through open-ended questions nevertheless 
show that the benefits provided by the VE experience are manifold, and are 
necessarily largely narrowed down by exclusively quantitative measures of 
matching categories between initial expectations and final outcomes. 


Our study evidences that students’ expectations are partly influenced by the 
objectives targeted in the different courses. Yet, the perceived outcomes 
also reveal gains in transferable skills (such as collaborative, relational and 
communicative skills) that are not always promoted in the course objectives. VE 
results in unexpected benefits, as it is very often a new experience for students, 
and very different from other pedagogical settings they are used to. 


Knowledge of students’ expectations and of unforeseen outcomes can be 
considered as crucial for educators and course designers in order to take better 
advantage of the learning potential of VE when defining the learning objectives. 
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In parallel, targeted learning objectives that embrace this potential are likely to 
help students to develop more fine-tuned skills. 
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